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ABSTRACT 

The Alaska State Operated School System 
now completed its second full year of experimental bilin 
education. In an area where Yupik Eskimo is often the fi 
language spoken by the native citizens, a decision was m 
experiment with teaching the early primary grades in the 
language, introducing English as a Second Language (ESL) 
manageable portions. To determine the validity of such a 
an evaluation design was conducted over a 3 year impleme 
period. The 2 categories of instruments used for the eva 
(1) academic — Yupik literacy and numerical skills; (2) 
linguistic — acquisition of gra'»=;mar and meaning in Yupik 
The analysis procedure was the method of t-test for diff 
between independent group means. The statistical analysi 
presented in 3 sections: (1) literacy, (2) numerical ski 
linguistic skills. In each section the results are gener 
described, followed by a brief discussion of the overall 
a whole. Program recommendations in the final section of 
concern: (1) instructional objectives, (2) English liter 
math, (4) alphabet, and (5) creative writing. (FF) 
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INTRODUCTION 



The Alaska State Operated School System (ASOSS) has now completed 
its second full year of experimental bilingual education in Yupik. Beginning 
in 1970 with a single kindergarten classroom and expanding co six schools 
the following year, children in thirteen Bristol Bay and Lower Kuskokwim 
village classrooms have experienced this new approach to their education. 

In an area where Yupik Eskimo is often the first and only language 
spoken by its native citizens, a decision was made to experiment with teaching 
the subject matter of the early primary grades in the Native language, intro- 
ducing English as a second language in small, manageable portions. This 
approach contrasts v/ith traditional schooling in the remainder of the area 
in which virtually the entire school day is taught in English. ^ 

In order to determine the validity of such an approach an evaluation 
design was established to be conducted over a three year implementation 
period. The goal of this evaluation was to assess by objective means the 
overall impact of the program in selected performance domains. Evalua- 
tion in the first two program years focused on determining the extent to 
which general language changes were taking place in the bilingually taught 
children. Since vocabulary growth is an important index of such charges 
tests were developed to measure Yupik and English vocabularies of children 
in the newly implemented programs. These scores, obtained at the beginning 
and end of the school year, were compared with those obtained in matched 
Bureau of Indian Affairs schools in which traditional programs were being 
carried out. 

^Excepting seven lower Kuskokwim villages engaging in similar bilingual pro- 
Q grams conducted by the Bethel Agency of the U.S. Bureau of Indian Affairs. 
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It was found by the end of the school year that children in the bilingual 
schools had gained not only* more Yupik vocabulary as was expected, but had 
also gained significantly more English vocabulary , even though English 
instruction comprised only a fraction of the school day. 

In the second year of the program, the number of schools participating 

at the first grade level was expanded to six and (or the first time a broad 

base of data would be abailable for evaluation. Again, relative vocabulary 

gains comprised the main evaluation focus. While the first graders showed 

the same significant gains shown by BIA bilingual first graders the year 

before, the BIA second grade students did not perform as well as their counter 

I 

parts in comparison schools. This reversal led to a good deal of speculation 
a,s to whether the ASOSS program would show a similar drop in performance 
when the second grade was added. For a number of reasons, the second 
year reversal, seen in the BIA program, was concluded to be the result of 
several factors, chief among which were (a) the wearing off of the initial 
novelty of the approach, and (b) the problems of preparing for and adding a 
second program level. 

The present evaluation design for the present year has data relevant to 
this question, and among other things, examines whether the second level 
problem is a result of its having been a new addition or if t^.iere are more 
deep-seated questions raised about the validity of the bilingual approach in 
general. If the second grade in this year's program shows relatively poor 
performance, real cause for concern might be warranted. On the other hand, 
if there is relative improvement at the second level the general health of 
tiie program will have been substantiated and theories that the addition of a 
new level will always presage a transitional setback will be discounted for 
the ASOSS situation. 



Needless to say, the first two years of the bilingual experiment demon- 
strated a mixed performance picture from which only incomplete conclu- 
sions could be drawn. Whether the momentum in language gains consistently 
shown by the first graders holds for the upper primary grades is the focus 
of a broadened third year evaluation design reported presently. 

Past Evaluation Designs 

The reader should bear in mind that the main features of the evaluation 
activities have been shared by both of Alaska's education agencies conducting 
bilingual programs in Southwestern Alaska; the Bureau of Indian Affairs (BLA) 
and the Alaska State Operated Schv.ol System (ASOSS), This arrangement 
has allowed a broader based opportunity to establish comparison groups from 
which to obtain comparative data. The procedures (though not the results) 
described here represent cooperative effort between the two agencies which 
has evolved over the past three years of program operation. 

As noted above, the first and second year evaluations included a focus 
on general language changes in the bilinguaUy taught children. In addition, 
measures of non-verbal intelligence were used to asses if other domains of 
performance could mediate changes in the behavior of the children. It was 
soon discovered that additional testing time involved with the latter measures 
did not warrant their continued inclusion in the evaluation design because 
their adaptation to a cross-cultural setting left them open tc ambiguous in- 
terpretations. 

The first year testing was conducted by a trained testing team who 
traveled to each bilingual and comparison village. In the second year, however, 
testers were trained in each of the program villages , eliminating the travel 
expenses incurred by a tq^sting team. The comparison data for the second 
year consisted of data obtained in each program village from students in the 
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upper primary grades, not yet included in a bilingual program. In most 
ways, this procedure came to be considered unsatisfactory from the stand- 
points that (a) objectivity was somewhat compromised by having teachers 
1 test their own students, and (b) diffusion of bilingual program effects from 

the lower to upper grades could not reasonably be controlled for statistically. 

The evaluation design for the present year has thus evolved from sub- 
stantive as well as logistical findings of the first two years and has, there- 
fore, changed in major ways. First, the number of performance skills 
measured has been expanded to reflect the need for specific information 
sought by program officials. Second, sources of objectivity compromised in 
the second year evaluations have been restored by re^urning^to the testing 
team concept. Third, rather than attempt to test all of the children in 
seventeen target villages (ASOSS, BIA and comparison) a stratified random 
sample hail been selected to minimize the loss of classroom instruction 
time for testers as well as students. And fourth, the testing has been limited 
to a single post test period, since the degree of initial comparTibility of 
comparison and bilingual schools had been satisfactorily established in the 
prior evaluation years. 

THIRD YEAR EVALUATION DESIGN 

Instruments 

The instruments used for the present evaluation fall into two main cate- 
gories, (1) academic Yupik literacy skills, and numerical skills; (2) 
linguistic acquisition of grammar in Yupik and English and acquisition 
of meaning in Yupik and English. 
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Comparison data for the first grade children consisted of data gathered in 
the comparison schools the previous year, 
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Academic Instruments —Yupik Literacy Skills 

The measurement of Yupik literacy skills was divided into three main 
catagories: (1) prereading , (?) decoding , and (3) encoding skills . The 
measurement of prereading skills consisted of the (a) recognition of initial 
letter sounds using stimuli such as: 




and directions {in Yupik): 

. Examiner points to initial letter '^d'* and says; 
^^This makes the sound "p'\ 

The examiner then points to each of the three pictures and says: 

^^One of these things begins with the sound "p^^; which one is it ? " 

If the child points to the correct one {no. 1) say: 

"Very good! "pitegcaun'^ begins with the sound ^^p^^ 

If he doesn't point to the correct one, point to the one he chose and say: 
"What do you call that? 

I£ what he calls it begins with *'p", say: 

"Very good! And this one (pointing to pitegcaun) also begins with 
"p", doesn^t it? " 

If it doesn't, go back to the beginning, and this time name ^he pictures 
for him. Do this as often as needed for the child to get the idea. 

Scoring: 

^ The number of pictures the child selects correctly; 



followed by similar stimuli and appropriate directions, (b) the visvial dis • 
crimination of symbols using the stimulus: 



Demonstration: 

and items: 

pi 
ta 
mu 



at 



atam 



tupik 
tema 
maraq 



tupa 

tuma 

murak 



qupellrua 

atata 

nutek 



with directions: 

Line up the words to the child's point of view. 

Point to demonstration item ^^AT**, and say: 

^^These letters can be found over here. Point to the place over here 
where you can see these letters together* " 

If the child points appropriately, say: 
^' Very good! You found them." 

then hand the child a pencil and say; 

*'Now make a circle around the letters.^' 

If he doesn't point appropriately, you point them out for him, give him a 
pencil and say: 

^'Now make a circle around the letters we found. '' 

Go on to the next one; ''PI, "TA", and "MU". 

and (c) reading phonemes using the stimuli: 
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c 

V 

* 

1 

and directions: 



y 

u 



r 
k 



Point to each phoneme and say: 

"What sound does this make?" 

Scoring: 

Write on the answer sheet a plus if the child says it correctly, or 
the sound the child makes is incorrect. 
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Stop after three failures in a row. 

The measurement of Yupik decoding skills consisted of 'a) reading sight 
words with the stimuli: 

tai-tai una neri ayii 

waniwa atraa qimugta 

kuimaa. mayva ermia 

and directions: 

Point to TAI-TAI in the book and say: 

"Here are some words from your reading books. Let^s see how 
many you know. Start with this one. 

Scoring: 

Write down the word the child says or a plus if he gets it right. 
Stop after three failures in a row. 

(b) Decoding new wordg ^ using the stimuli: 

patu amci qilak 

qalu qamiquq uqilauq 

and directions: 

Point to ^'patu" in the booklet and say: 

'^Here are some new words. How many can you figure out? 
Start with this one. " 

Point to and sound out "patu. " 

Go on to the rest of the words and give no further help. 
Scoring: 

Write down the child's decoding attempts or a plus if it is correct. 
Stop after two failures in a ? ow. 

(c) matching words with pictures, (see Appendix for stimulus material) with 
the foUov/ing d .sections: 



In the latter case, children in comparison schools were r;iven full credit 
even if they gave an appropriate English response to a particular phoneme. 
In general, instructions were appropriately modified to adapt to the com- 
parison school environment. 
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Point to **ENA^' on the answer sheet and say: 

"This word says ENA and here is a picture of ENA. See how I 
draw a line to connect them. These other words have pictures 
too. Draw a line to connect the words to each picture. " 

and (d) reading and following simple directions, with the stimuli: 
Demonstration: quuyurni 

aqximi natermun qengan enirru 

amiik ikiresgu put'en 

qeckaa , una ;en elliu qamiquvnun 

ajid the directions: 

Point to "quuyurni'^ in the booklet and say: 

"This says to do something. It says "quuyurni.^' (demonstrate for 
him). 

Point to the rest of the directions and say: 

"Each of these says to do a different thing. Read each one out loud 
and do what it says to do. Start with this one. ^' (Point to the next 
direction). 

Scoring: — ^ 

Score 0 if he doesn't read the direction. 

Score 1 if he reads it but doesn't do what is asked. 

Score 2 if he does what is asked. 

Stop after he misses two in a row. 

Thus, the measurement of decoding skills is designed to assess not only 
simple decoding performance as in (a) and ^^)), but also his ability to attach 
recognition meaning to the symbols he decodes, as in (c) and behavioral 
meaning as in (d). 

Finally, the measurement of encoding skills assesses three levels of 
written performance: (d) ability to write the alphabet (appropriate to Yupik 
or English), (2) ability to encode Yupik soimds and word s, and (3) per- 
formance in which the pupil writes about himself> 

In writing the alphabet the following directions were used: 



V 



8 



Point to the place on the answer sheet where the child is to write the 

Yupik alphabet and say: 

'^Here is some paper to write the letters of the Yupik alphabet 
(or English if control school). You know A, C, E. (or A, B, C, 
f or English). Start here and write them all. " 

Scoring: 

1 point for each correct letter (appropriate to the alphabet the 
child attempts; Yupik or English). 

0 points for each repetition. 

0 points for reversed or poorly aligned letters. 

Divide the total points by 15 (Yupik) or 26 (English) and multiply by 
100 to get the per cent correct* 

The next level of measured encoding skills consisted of the child's ability 
to encode correctly Yupik sounds (phonemes) and words given the following 
stimuli: 



1. 


pa 


8. 


JIU 


15. 


nuna 


2. 


pu 


9. 


Vina 


16. 


patu 


3. 


tu 


10. 


ani 


17. 


unan 


4. 


ta 


11. 


ata 


18. 


tauna 


5. 


ci 


12. 


ayii 


19. 


nutek 


6. 


ca 


13. 


qaku 


20. 


camek 


7. 


ni 


14. 


tuma 


21. 


panik 



and directions; 

Point to the place on the answer sheet for writing sounds and words and 
say: 

"I want you to write down (point) here the sounds that I say. Let^s 
start with "pa'\ etc. 

Scoring: 

1 point for each corrected spelling divided by 21 and multiplied by 
100 to give the percentages. 

Stop after four in a row are incorrect. 
Finally, the highest levels of incoding performance measured in the pre- 
sent evaluation consisted of eliciting a written essay from the child in the 



Words properly encoded in English were counted correct in comparison 
schools. 



following manner: 

Point to the appropriate space on the answer sheet and say: 

"Now I want you to write something about yourself. 

Write anything you want and write as much as you 

want. You could write about where vou live or vour 
.. - - ■ ■ . — , i 

family or about tilings you like to do.'' 
Allow 10 to 15 minutes for the child to finish, alone if possible. 
Scoring: 

Score 1 point for each word written. 

Score 1 point for each correctly spelled word. 

Then divide the total number of correctly spelled words by 

the total number of words. 

The child was allowed to respond in which ever langmge he was most com-* 
fortable. 

As can be imagined, the initial scoring of the essays posed some pro- 
blems, chiefly in making cross -language comparisons in an objective manner. 
The final scoring was accomplished by the following steps. 

Step One: All responses were translated into a common 
language --English- -by experienced Yupik teachers. In 
all cases errors of grammar and syntax were translated 
as faithfully as possible, Nonsensical responses were 
noted as such, 
'step Two: All responses were then transcribed to 3x5 
cards by a third party with the child* s name, school, and 
grade level on the reverse side. 

Step Three: Three e^cperienced rural Alaskan primary 
teachers independently rated the responses on a seven 
point rating scale, basing their judgements on the overall 
quality of the written content, disregarding spelling errors. 
Step Four: The independent ratings were combined to 
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give each child a total essay score. 



Academic Instruments - -Numerical Skills 



Measurement of numerical skills consiste.d of two main components. 
The first focused on the ability of the children to use names of numbers 
' for countinjg and the second focused on the ability of the children to perform 
a variety of arithmetic calculations. 

Counting skills were assessec in three parts; (1) ora "^ countings using 
the directions ; 

Counting from 1 to 30 in Yvxpik (or English in comparison schools). 
Say to the child, '^Now I would like yovi to count from 1 to 
30 in Yupik (or English in comparison schools). 

Scoring: 

Scorii 1 point for 3ach correct number. 

The score was the highest number up to 30 the child could 
count without error. 

(2) recognition of numbers, with the stimuli: 

80237152964 
56 75 42 64 57 13 35 
178 124 281 339 

and directions : 

Point to the number and say: 
"What number is this?^' 

Same directions or the second row of numbers except discontinue 
after three faibires in a row. 

And the same directions for the third row except discontinue after 
two failures in a row. 

Scoring: 

Score 1 point for each correct number. (Note that two and three 
diget numbers are only single numbers, for example 56 is not 



As an estimate of the reliability of the three judges ratings, the patterns of 
these agreements over the 189 responses were as follows: All three agreed: 
Q 126 (66.7%); two agreed 62 (32.8%); all three disagreed 1 (less than 1%), In 
j^D J(^" only two cases did disagreements span more than two scale points* For all 
yn^ta possible judgements there was 89% agreement. 
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5--6, (five--six) it is fiity-six. ) 

and, 

(3) counting objects, with the stimuli sho^^^>: in Appendix A, 
and directions': 

Say to the child, ^'Now I would like you to count these things 
and tell me how many of each there are. Start with these. 
(point to the triangles). 

Scoring: 

Score 1 point for each correct answer. 

The second component of numerical skills measurement consisted of 

basic arithmetic as follows: 

The tester writes out each problem on the child's tablet (one at a time) 
and says: ^^You take the pencil and do this one. After he does the 
first one, write out the second one and so on. 

Scoring: 

Score 1 point for each correct answer. 
Addition: 

3 1 6 6 4 11 36 104 

+ 2 +3 +3 ±5_ 5 + 5 + 15 + 9_ 

+ 1 

Discontinue after three failures in a row and go on to subtraction. 
Subtraction: 

4 5 8 8 56 41 232 332 
■1 -3 -2 -4 "5 - 7 - 21 - 25 



Discontinue after three failures in a row and go on to multiplication. 
Multiplication: 



1 
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Linguistic Instruments 

For the broad purpose of assessing comparative language changes in 
program and non-program children, two subtests of the Illinois Test 
Psycholinguistic Abilities (ITPA) were adapted, - - The Grammatic Closure, 
and the Auditory Association subtests. According to Kirk et al, (1968)''' 
the Grammatic Closure subtest: 

...assesses the child's ability to make use of the redvindancies 
of oral language in acquiring automatic habits for handling syntax 
and grammatic infTections. In this test the conceptual difficulty 
is low, but the task elicits the child's ability to respond automat- 
ically to often repeated verbal expressions of standard American 
speech. The child comes to expect or predict the grammatic 
form so that when part of an expression is presented he closes 
the gap by supplying the missing part. The test measures the 
form rather than the content of the missing word, since the con- 
tent is proyided by the examiner, (p. 11) 

The Auditory Association subtest: 

...taps the child^s ability to relate concepts presented 
orally. In this test the requirement of the auditory receptive 
process and vocal expressive process are minima? r while the 
organizing process of manipulating linguistic symbols in a 
meaningful way is tested by verbal analogies of increasing diffi- 
culty. A sentence completion technique is used, presenting one 
statement followed by an incomplete analogous statement, and 
allowing the child to complete the second statement appropriately, 
(p. 10) 

These two tests, as published, are designed to deal with verbal output 
at two different levels of language organization. By adapting the stimulus 
material to the familiar locale of the Eskimo child, and adapting by trans- 
lation the verbal content of each test item it was hoped that similar pro- 
cesses would be measured in children affected by the bilingual education 
program. To be sure, the difficulty of achieving a perfect adaptation of 



Kirk, S.A. Mc Carthy, J.J. and Kirk, W.D., Examiner's Manual: Illinois 
Test of Psycholinguistic Abilities, University of Illinois Press, Revised 
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both the visual sfimulus material and the verbal item content is great and 
as many steps as possible have been taken to assure appropriatences 
within the given situation. 

The test adaptations were made in conjunction with personnel of the 
Eskimo Language Workshop, whose task it was to (a) modify test pictures 
to the local environment, (b) translate item content into meaningful tests 
of grammatical structures, and (c) provide back-translations for use ?.n 
corresponding English language items. In most cases the English version 
is not a direct literal translation of the Yupik, but has been readapted to 
make the syntax me aningful as a test item. 

The linguistic tests were administered as follows: 

1. Grammatical closure — each item has a stimulus picture and oral 
verbal statement, beginning with a demonstration: 

"Waniwa-llu malruk . " (pointing to the picture of two beds, 

and gesturing for the child to complete the sentence). If the child doesn't 
respond he is given the answer and the item is repeated until the child gets 
the idea before proceeding to the first item in the 32 item set. The same 
procedure is used in the English version of the test. 

2, Auditory Association--each item represents a verbal analogy which 
the child must solve by giving orally an appropriate completion to the sen- 
tence given by the administrator. The test is begun with a demonstration 
item: *'Aataq ang'uq, piipiq . (father is big, baby is . ) 

followed by the reverse: "Piipiq mik'uq, aataq . " When the child 

shows he has the idea, the tester proceeds through the remaining 38 items. 

For both the Grammatic Closure and Auditory Association tests, the 
Yupik and English versions are administered seperately. 

Table 1 summarizes the preceding description for quick reference by 
the reader. Included in Table 1 are the combinations of subtest components 
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used in the final statistical analysis. For example, a total pre-reading 
score was obtained by combining its three subtest components; initial 
letter sounds, visua; discrimination of symbols, and reading phonemes. 
In most cases, subtests were combined where it would ease the burden 
of statistical calculations, provided there was reasonable homogeniety of 
content. In cases where subtests are not combined it was felt the sub- 
tests either were measuring divergent skills {e.g. , numerical skills) or 
used measurement scales too varied to permit combination without under- 
going time consimning statistical scale transformations (e. g, , encoding). 
The resulting combinations of tests and subtests siimmarized in Table 1 
provide a total of eleven units for statistical analysis. 

Testing Procedures 

All tests were administered by experienced Yupik bilingual teachers 
recruited from the ASOSS and BIA bilingual programs. Testers were 
selected according to four major criteria: (1) personal interest in the 
testing program, (2) recommendation by principal teachers involved in 
the program, (3) availability for travel to a training workshop, and (4) 
assent by the majority of bilingual Aides. Of the eight selected, three had 
prior experience as testers in earlier evaluation activities. 

The testers received the main portion of their training at a three-day 
workshop held in early March at the Bureau of Indian Affairs site in Bethel. 
During the three days, general testing concepts such as measurement and 
random sampling were assimilated as well as specific administration pro- 
cedures* In addition, the testers gave substantial input into the final struc- 
ture of the tests, developed scoring criteria, and laid the groundwork for 
the math test to be used in the program. 

Following the workshop, final production of the tests was completed, a 
final testing schedule was developed, and a random sampling plan was finalized 



TABLE 1 
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SUMMARY OF EVALUATION TESTS BY GRADE LEVEL 







Number of Subjects 
by Grade Level ^ 


Ability Tested 


Instruments 


Level One 


Level Two 




^- 




Ril 




Ril 
JD IJL • 


C omp « 




Literacy Skills; 


prereading 

initial letter sounds 

+visual discrimina- 
tion of symbols 




- 










+ reading phonemes 


30 


19 


30 


19 




Decoding: 


reading sight words 

1 \JL \^ ^ \y Vl, X H,pL W W \^ X VX 1^ 

with pictures 














+reading and following 
directions 


r 

30 


19 


30 


19 




Encoding: 


alphabet 


30 


19 


30 


19 






sounds and words 


30 


19 


30 


19 






free essay 


30 


19 


30 


19 




Number Skills: 














Counting: 


oral counting 
•f naming numbers 
















30 


19 


30 


19 




Arithmetxc: 


arithmetic 


30 


19 


30 


19 




Linguistic Skills: 














Grammatic 
Closure: 


Yupik 


30 


19 


30 


19 






English 


30 


19 


29 


19 




Auditory 
Association: 


Yupik 


30 


19 


30 


19 






English 


30 


19 


30 


19 





O T'Because the kindergarten pupils have no appropriate comparison group, the 
sl^Cmalysis of their data is not included. 
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including the village assignments for each tester. Travel arrangements 
were coordinated locally by BIA and ASOSS area administrators. Appendix 
B contains the finalized adininistration instructions, testing assignments, 
and sampling procedures for all posttesting activities. With few exceptions 
the testing program was carried out satisfactorily. The few exceptions 
were the result of unforseable local conditions requiring on the spot decisiors 
by the particular tester. In only one case (see Table 1, footnote) was there 
a significant loss of data, but even then the ability to draw data-based con- 
clusions was not seriously jeapordized. 

RESULTS AND DISCUSSION 



The analysis procedure used throughout was the method of t-test for 
difference.""^ betw^-en independent group means. ^ To explain further, the 
t-test provides an estimate of the probability that two group means could 
differ an observed mount simply by chance. A decision can thus be made 
whether or not to place confidence in the effectiveness of a program. This 
is done by rejecting or not rejecting the idea that a particular test result, 
comparing a group of program with a group of non-program children, could 
have happened as a result of chance or luck. For example, if a t-test shows 



The t-test for independent means proceeds as follows: 

Where: Xs a mean score for a bilingual group 

5^ is a mean score for a comparison group, and 
Sj^j^ is the standard error of the mean difference, estimated from 
the two sample variances. The resulting t value is compared with tabled t 
values for various sample sizes. For a more technical treatment of the t 
statistic the reader is referred to Edwards, A. L. , Experimenta l Design in 
Psychological Research, , Holt, Rinehart and Winston, New York, 1963, 
Chapters 7 and 8. 



that a difference between two group means could be expected to happen by- 
chance not less than twenty times in a hundren (i. e, , with a probability (p) 
greater than . 20) we would fail to reject the notion that the difference was 
due to chance and thus have little confidence in the idea that the program was 
effective. If, however, a t-test shows that the means could differ by chance 
fewer than five times in a hundren (p . 05) we will have reached the commonly 
accepted scientific standard for rejecting the idea of chance differences and, 
therefore, be able to have confidence that the program was indeed effective. 
Of course, the same decision rules hold for cases in which the comparison 
group does better than the program group. Such cases are shown in the 
results as negative t-test scores. 

The results of the statistical analysis are presented in three main sec- 
tions: (1) literacy skills, (2) numerical skills, and (3) linguistic skills. 
In each section the results are described generally, followed by a bried dis- 
cussion of the overall patterns in the results taken as a. whole. Program 
recommendations are given in the final section of the report. 

Literacy Skill s 

Table 2 shows the results of the statistical analysis of performance in 
literacy skills for er.ch grade level. The reader is reminded that the values 
for t are the best index of comparative performance since they indicate 
whether a particular mean difference between a bilingual and comparison 
group should be taken seriously, i.e. , represents a significant program 
difference. Negative t values indicate a higher comparison group mean. 

Beginning with prereading skills the performance of the bilingual pro- 
gram children was substantially superior. This superiority held for both 
grades one and two. The important implication of this high performance 
is the evidence it gives for a good beginning in Yupik literacy, comparable 
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to what might be. expected by. the third year in the traditional program where 
the children must rely on their ability to generalize from what they have 
learned in English iiteraqy training. In Yupik decoding skills, the bilingual 
program children show clear superiority at each grac^'" level. 

In encoding, a mixed picture of performance has taken shape. While 
the ability of the bilingual program to establish the concept of the written 
alphabet is no better than in the comparison program, the ability of the 
children to encode Yupik sounds and words successfully is quite strong at 
every level. R.elative skills at free written expression is quite strong in 
both levels of the program, especially in view of the fact that the comparison 
children were allowed to express themselves in English. 

The reader should bear in mind that the purpose of evaluation in this 
section was to assess the degree to which the bilingual classroom is able 
to prepare Yupik speaking children to be literate in their first language. 
Using the traditional classroom as an estimate of what might have happened 
otherwise, makes relatively clear the general success in meeting this in- 
structional goal. The only minor exception, writing the alphabet, was one 
of the two areas in which children in comparison schools were not restricted 
by the tests from relying on English as a mode of written expression. In 
all other cases virtually no generalization from English to Yupik was in 
evidence by children in the traditional program. It would, of course, be 
unfair to say that no literacy skills in English are being developed in the 
traditional schools since the evidence certainly does not support such a con- 
clusion. 

It does seem certain that a concept of the alphabet is not necessary for 
other basic encoding operations particularly in the accurate formation of 
sounds and words received aurally. The bilingual children do fine without 



20 



it and the traditionally taught children are at no apparent advantages possessing 
it. (recall that spelling was not considered in judging the free essay). Per- 
haps teaching an alphabet is more for the reinforcement of the teacher than 
of the pupil and hence constitutes an unneccessary part of the curriculiim. 
In fact, trying to establish an alphabet concept early may only lock the child 
into an ungeneralizeable system which later the child is required to repudiate 
upon second language literacy training. 

Generally it appears that a sound basis in literacy skills in being devel- 
oped in the bilingual program. 

Numerical Skills 

Table 3 shows the test results for assessing comparative numerical 
skills. Two components were tested; a component comprision counting and 
identification (naming) skills, and a component comprising common arith- 
metic calculations. In the former, counting, the comparison groups performed 
better (though not significantly so) than their bilingually taught counterparts 
at both grade levels. However, in arithmetic the bilingual program children 
performed substantially better than the comparison school children. 

The problem of establishing a Yupik math curriculum has been present 
since the program^ s beginning. First of all, there exists no standard treat- 
ment of math throughout the bilingual program schools. For example, pro- 
gram schools vary in the time at which English names for numbers are in- 
troduced. Second, most Yupik counting systems are developed on a metric 
using the base twenty, necessitating highly complex transformations into 
the English system of base ten. For numbers below 20 or 30, there is gen- 
erally no difficulty but numbers greater than 30 begin to possess long and 
cumbersome names, which are linguistically, as well as mathematically 
different from English equivalents. 
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Why then should arithmetic pose no apparent problem given the counting 
difficulty? One possible reason is that the arithmetic problems used in the 
present evaluation are, like most arithmetic operations, approachable by 
reduction to single integers. In fact, even into secondary school most math 
calculations are performed by reduction into single digit operations. This 
may be why the children in the bilingual program can handle calculations 
reasonably well without the apparent facility with large number concepts. 
However^ it follows that when such concepts become necessary at some later 
time the children in the bilingual program may well have problems developing 
the necessary abstractions to go forward. This evidence calls for serious 
consideration to redesigning the entire Yupik math curriculum; from materials 
development to teacher preparation, 

Lfinguistic Skills 

Linguistic skills in each language were measured in two ways, one 
stressing the acquisition of grammar and syntax and the other stressing the 
development of meaning. 

Referring to Table 4, in Yupik the quality of performance in grammatical 
use is clearly greater at grades one a.nd two for the bilingual program students. 
In their ability to deal with meaning in Yupik, the bilingual children show 
significantly better performance at both grade levels. 

In English grammatic development there is a strong beginning performance 
by bilingual program children which is maintained into grade two. More im- 
portant, the development of facility with meaning in the English language is 
clearly greater in the bilingual program than in the traditional schools. 

It should be noted that of the two English tests, Grammatic Closure 
measures the abilities most stressed in the English as a second language 
(ESL) portions of the bilingual program curriculum, and this is where the 
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stronger performance was especially apparent in early grades. It is rea- 
sonable to predict that the early basis in English grammatical use and asso- 
ciative meaning and all aspects of language development in Yupik will show 
a multiple effect when increased exposure to English takes place in the post 
primary years. 

General Discussion 

The foregoing results give substantial evidence that the ASOSS bilingual 
education program is having a strong positiv . impact on the children it serves. 
With minor exceptions, growth seems to be taking place in academic subject 
mastery as well as in bilingual oral language development. The program's 
broad goals are being met in all areas of first language advancement and 
evidence is strong that second language learning is taking place at a rate 
well beyond what might have been expected under a more traditional education 
program. 

But with this general success it is necessary to note certain general areas 
in which further program development might be entertained. 

PROGRAM RECOMMENDATIONS 

1. Instructional Objectives, Even though the evidence points to general 
program success the evaluator^s opinion is that renewed effort should be made 
by program administration to develop localized, individualized instructional 
objectives. The present lists of objectives reflect only the ideas of persons 
responsible for writing the initial program proposal, none of whom have 
been involved in the program's operation subsequent to it^s beginning imple- 
mentation three years ago. These objectives, because they reflect neither 
the thinking of those currently responsible for the program operation now of 
the native communities the program serves, have little value as instructional 
guidelines or as operational norms for program evalxaation, 

ERLC 
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And there are other problems associated v/ith the present list of objectives, 
i'irst, they are generally not stated in measurable terms, for example '^knows 
that all children everywhere do not dress, live, eat or act just as they do, ^' 
or '^develop pride in legible writing in Eskimo, " and thus do not meet the 
basic criteria for use in gaining knowledge about a particular pupil's pro- 
gress. Second, they are not specifically associated with instructional processes 
designed to carry them out. For example, in at least two villages, the ob- 
jective "draw a simple map of the village, showing main buildings and direc- 
tions" was not even attempted, so to judge the accomplishments of the children 
on the basil of whether they did indeed produce such a map would be unfair. 

Suggestions for developing a solid foundation of instructional objectives 
have been outlined in earlier evaluation reports and need not be repeated here. 
The evaluator strongly recommends that some plan be developed along those, 
or other lines so that future program operations and evaluation will be served 
by a systematic approach more reflective of local prograrr. intentions. 

2. English literacy training . One of the more interesting developments 
in the past year wr.s the desire of some second language teachers to begin 
English literacy training in the second grade, in advance of the original pro- 
gram design to wait until later (preferably late third or fourth grade) to do 
so. The rationalization given by the teachers was that one or two children 
"seemed to be ready, " i. e. , spontaneously attempting to read English ma- 
terial (book titles, etc. ). Although strongly opposed by the evaluator, the 
director of Yupik materials development, and the developer of the ESL com- 
ponent, the program director gave official sanction on the rationale that to 
fail to go ahead with English literacy training for these few children would in 
some way "hold them back." There is now evidence that at least four villages 
have gone ahead with substantial English literacy training programs of their 
own, effecting nearly half the second grade children. The evaluator considers 
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this to be a major modification in the program's original intent, and testimony 
from second language teacher* s responsible for ESL training in the early- 
primary grades indicates a serious weakening of morale taking place insofar 
as their efforts to begin oral language development through ESL are not 
being continued at the later stages. The evaluator recommends a complete 
review be undertaken by program management concerning this situation be- 
fore further erosion of the total bilingual program produces language develop- 
ment gaps too wide to mend. 

3. Math, The bilingual approach to math should be reviewed with a basic 
goal of finding some sort of compromise method of identifying numbers in the 
classroom. It has been suggested in numerous quarters that all phases of 
rnath use English names for numbers so that the base ten counting system 

can be established without linguistic confusion. Math ilself, should not rule 
out teaching counting in Yupik in the beginning grades, but this should be 
treated, rightfully so, as a part language development and cultural enrich- 
ment, and not a vehicle for learning math. 

4. Math materials. Another attempt should be made to develop math 
materials in the Eskimo Language Workshop. But, rather than adapt com- 
mercial textbook approaches, such as the Addison V/esley series, an effort 
should be made (1) to secure a math consultant to work with the Eskimo 
Language Workshop, and (2) develop a program of on-site implementation 
to help bridge the gap between newly developed math materials and their 
implementation in the classroom* 

5. Alphabet. Review the alphabet concept and the role it plays in de- 
veloping encoding skills. In reviewing the responses to this test item, the 
biggest problem seems to have been the intrusion of letters from the English 
alphabet into the thinking of the bilingually trained children. As stated 
before, this appears to have had little adverse effect on the bilingual program 
children since intrusions of English letters when sounds and words were en- 
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coded happened rarely. These problems are definitely more acute for the 
comparison school children who seem not to be able to seperate their lan- 
guages cognitively supporting the validity of the bilingual approach where 
this kind of confusion apparently is being circumvented. 

6. Creative writing. It is clear that more time net '. 3 to be devoted to 
creative written expression in Yupik in the upper primary grades. There 
can be little doubt that the children in the bilingual program are gaining the 
basic encoding skills for writing but these skills may well be lying fallow. 
This problem raises the broader question concerning present limitations 
of (bilingual teacher) training posed on the bilingual teacher. Given the 
present pre-service preparation plus short in-service workshops it is im- 
pressive indeed that so much has been accomplished. The time is ripe for 
the development of a stronger in-service training program that will free both 
the bilingual and the ESL teacher to become resources to one a.nother, with 
the help of outside guidance from program administration and their consultants 
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APPENDIX A 
Stimulus Material for Yupik Literacy Tests 
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ANSKER SHEET A 



SCORE 



VISUAL DISCRIMINATION Of SrMBOLS 

demonstration: 

at atam 



pi tupik tupa qupellrua 

4 



ta ■■ tema tuma atata 



mu 



maraq 



murak nutek 




yaquiek 



APPENDIX B 
Instructions to Testers on Testing and Sampling 
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CENTER FOR NORTHERN EOUCATION 




University of Alaska 



COLLEGE. ALASKA 99701 



March 28, 1973 



Dear 



Here are some instructions ^or you to follow in doing the testing. 
By now you should have received the testing materials, or if you haven't, 
you should shortly. 

When I got back to Fairbanks after our workshop and started working 
out the details the first thing I noticed was that some of you would have 
had a much more difficult job than others, so I went ahead and made a few 
changes to help even out the number of children each of you would test, 
I hope you won*t be mad at me. I also found that the Math Test, as we 
originally worked it out in Bethel was too long so I shortened it a little 
bit and simplified it*s administration (I hope). *• 

If you look in Table I on the next page, you wifl see the lists of 
schools to be tested and who will be testing at each school, I also put 
down the number of children to be tested at each grade level. Following 
Table I is a detailed procedure for getting a random sample of children 
for testing. Go over it and practice it until you understand it. This 
is very important. 

Also, when you test the children, start with Grammatic Closure and 
Auditory Association in Yupik. Do the rest of the tests in whatever order 
you think is best. Break up the testing into at least two sessions for 
each child, maybe even three, so he won't get too tired. 

Before you go out to the other villages, you should practice all of 
the tests on a couple of your own pupils. To do this, you can first figure 
Out which children in your village are going to be tested, (this will be 
good practice itself) and then choose a couple of children you know will 
not be tested to practice on. I know this all seems complicated, but if 
you do enough practicing before you go out, you will do a lot better job. 
Also, make sure the other teachers know what you are doing so they will 
understand too. 
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TABLE I 

« 

Testing assignments and numbers of children to be tested in BIA, ASOSS 
and comparison schools, April 15-30, 1973. 



SCHOOLS TESTERS NUMBER TESTED (BY GRADE) 







K 


1 


2 


3 


TOTAL 


COMPARISON SCHOOLS 


Kwigill ingok 


Frank Mathev/ 




5 


5 


4 


14 


Eek 


Sophie Parks 




5 


5 


4 


14 


Napaskiak 


Sophie Parks 




5 


5 


4 


14 


Kwethluk 


Tim Samson 




5 


5 


4 


14 




SUBTOTAL 


0 


20 


20 


15 


56 


BIA SCHOOLS 


.Kipnuk 


Lucy Nelson 




5 


5 




10 


:Quinhagak 


Lucy Nelson 




4 


4 




8 


Tuntutul iak 


Lucy Nelson 




4 


4 




8 


.Napakiak 


Tim Samson 




4 


4 


5 


13 


Kasigluk 


Helen Nicori 




5 


5 




10 


Nunapitchuk 


Zack Ivon 




5 


5 


5 


15 


/(kiachak 


Helen Nicori 




4 


4 


5 


13 




SUBTOTAL 


0 


31 


31 


15 


77 


ASOSS SCHOOLS 


Bethel 


Frank Mathew, Zack Ivon 


10 


5 


5 




20 


Kongiganak 


Joe Alexie 




5 


5 




10 


Twin Hills 


Anecia Alakayak 




4 


3 




7 


Togiak 


Anecia Alakayak 




8 


8 




16 


Manokotok 


Joe Alexie 




7 


7 




14 


Aleknagik 


Jos Alexie 




4 


4 




8 




SUBTOTAL 


10 


33 


32 




75 




. GRAND TOTAL 


10 


84 


83 


31 


208 
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SAMPLING PROCEDURE 



In order to test a random sample of children at each site, the following 
steps should be taken. 

STEP I : Obtain an alphabetical list of boys and a list of girls for each 

■ grade level from the school's principal teacher and number the children 
on each list, starting at the top. 

STEP 2 : Remove from each list any children who have been absent from school 
more than ten school days in a row . 

STEP 3 ; Determine from the schedule of testing assignments, how many 
children in each grade you are supposed to test at that school, (for 
example, Lucy is going to Kipnuk and will test 5 level one children, 
and 5 level two children). 

STEP 4 ; Determine how many boys and how many girls you will test at each 
grade level so that the number of boys and girls is about equal. If you 
are supposed to test an even number, say 6, then test 3 boys and 3 
girls. But if you are supposed to test an uneven number, like 5, check 
which list is longer and select the extra one from that list. So if 
there are more girls than boys and you need to test 5 children at that 
grade level, test 3 girls and 2 boys. 

STEP 5 ; Use the following list of random numbers I've selected ( 6,3,9, 

7,1,4,10,8,2,5) to determine exactly which children on each list you should 

test. For example, Anecia is in Togiak and she is supposed to test 
8 children in grade level one. She will test 4 boys and 4 girls. 
Let's say that the girl's list has 8 names on it. She would then select 
the 6th girl on the list, then the 3rd on the list, (she can't select 
the 9th because there are only 8) the 7th on the list and the 1st on 
the list. Then she would repeat this step for the boys and girls in 
grade level two. This method will work for any size list up to 10, If 
there are more than 10 on the list, go to step 6. 

STEP 6 ; If a particular list has more thn 10 names on it, divide the list 

■ in' half and make your first selection from one list, the next selection 
from the other, and the next from the first list and so on. For example, 
let's say that one of your boys lists has 12 names on it. Divide it 
into two lists of 6 names each. The first child selected would be the 
6th name on the first list, the second child selected would be the 3rd 
name on the second list and the third child selected v/ould be the 1st 
name on the first list, and the fourth name on the second list. Get it? 

Do this until you have selected all the names you need from the list. 
Remember, do step 6, only if you have a list with more than 10 names on 
it.' Otherwise you only need to do steps 1 thru 5. 

Practice this in your own village with your principal teacher until you get 
used to it. You can impress him by explaining to him what a Random Number 
Table is. 
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Let me wish you the best of luck, and thank you for all the help you have 

given us so far. I look forward to hearing from you, and seeing your test 
results. Remember, be fair to all the children and above all be nice so they'll 

like you and want to do well for you. 




Sincerely. 



Jim Orvik 



JO/jf 



